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ACCELERATED LEARNING PROGRAM 
(ALP): GRADE 3-8 EVALUATION 
2001-02 

Abstract 

WCPSS initiated the Accelerated Learning Program (ALP) in J 999-2000 as the 
major new intervention to help all students reach grade-level performance in 
reading and math. Analyses of the program ’s third year results show that overall, 
changes in both achievement growth and performance support the effectiveness of 
ALP and other assistance at grades 3-8. 



Report Summary 

BACKGROUND 

For many years, the Wake County Public School System (WCPSS) has provided instructional 
support to students with special needs through a variety of grant- funded programs. However, 
student needs could not be fully met through these resources. To help accomplish Goal 2003, 
local dollars were allocated for supplemental instructional support through the Accelerated 
Learning Program (ALP). 

ALP began in 1999-2000; 2001-02 was the third year of the ALP program at grades 3-8. 
Initially, students received ALP instruction only outside of the school day to extend their 
instructional time. Over time, however, schools have been allowed to provide ALP during the 
day, if desired. Students who scored below grade level were to be offered up to 22 days of 
additional instruction through trained instructors (preferably teachers) in small groups (of 15 or 
fewer students) during the school year. Grades K-2 and 9-12 were added to ALP in 2000-01, as 
I well as a Summer Academy. These components are addressed in other Evaluation & Research 
Department (E&R) reports. 



ALP 2001-02 

The ALP program at grades 3-8 was funded through local and state funds. Funding formulas 
provided funding per child scoring below grade level (generally Level I or II on state End of 
Grade [EOG] tests) at all schools plus an allotment for schools with higher concentrations (over 
30%) of low-income students (called challenged schools). Schools were allowed to assist 
students who scored on grade level (Level III) through ALP, as space allowed, based on 
classroom performance. 
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The number of students in ALP in 2001-02 was 7,285, very similar to the 7,325 students served 
in 2000-01. However, the number of Level III and IV students (students scoring at grade level) 
participating increased from 502 to 1,654, meaning that the number of students served who 
scored below grade level actually decreased. The increase in Level III and IV students served 
could be exaggerated because of a change in data collection methods. Most of the Level III and 
IV students served in ALP were elementary students. Participation by school varied 
considerably for students scoring below grade level, from 22.2% to 99.5%. 

At both the elementary and middle school levels, ALP provided support in reading, mathematics, 
and writing. 

• The most common instructional approaches used were targeted instruction (small groups 
focusing on specific skills) and team teaching at the elementary level, with electives more 
common at the middle school level. 

• Most schools (53.3%) offered assistance in a combination of session times on school days 
and non-school days, with after-school, Saturday, and during-the-day sessions the most 
popular options at traditional-calendar schools (plus intersession at year-round schools). 

• Using ALP funds to provide students with additional support during the day has become 
more common over time, especially in traditional-calendar schools, increasing from 0% 
in 1999-2000 to 34% of the elementary and 29% of the middle schools in 2000-01, and 
42.5% of the elementary and 60% of the middle schools in 2001-02. 



Level I and II Trends 

ALP was the largest intervention effort in place at grades 3-8 designed to improve the 
performance of Level I and II students, and most students participated. Therefore, examining 
student growth for all students who scored in Levels I or II in spring 2001 provides useful 
information on overall program impact. Changes in both achievement growth and performance 
support the effectiveness of ALP and other assistance at grades 3-8. 

• Students in grades 3-8 scoring in Level I or II have shown improved growth since ALP 
began. Level I or II students showed high growth on the ABCs at both the elementary 
and middle school levels in both spring 2002 and 2001. Elementary students initially 
scoring at Level I or II also showed exemplary (high) growth in 2000. High growth is 
necessary for these students to reach grade level. 

• The percentage of Level I and II students able to reach grade-level achievement in one 
year has increased steadily since ALP began. Only 36% of students who scored below 
grade level in spring 1997 were able to score at grade level by spring 1998 (before ALP 

n). In contrast, of those scoring below grade level in spring 2001, 51% in reading 
53% in math reached grade-level scores in spring 2002. 




PROGRAM IMPACT 
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• The overall percentage of students at grade level is influenced both by students who 
move up from Level I or II to Level III or IV scores and those who drop from Level III or 
IV to Level I or II scores. The percentage of students tested in spring 2001 and 2002 who 
showed a decline to Level I or II was 3.9% in reading (about the same as spring, 2001) 
and 3.1% in math (an improvement over 2001). The net increase in students in Level III 
or IV was higher in reading (1 ,358) than in math (1 ,056). Net increases were higher in 
reading at elementary and math at middle school (see next figure). 

Net Increase in Students in Level III or IV 
by Grade Span, Spring 2001 to 2002 
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ALP Effectiveness 

ALP participants at both the elementary and middle school levels showed high growth in both 
reading and math based on the North Carolina ABC Accountability (ABCs) Program regression 
model. Subgroup growth was also strong. (One part of the ABCs utilizes regression analyses to 
examine student growth from one year to the next compared to the state.) 

• Nearly all subgroups met the ABC high growth standard in reading and math. 

However, ABC growth composites were higher for students not on free or reduced- 
price lunch (FRL) than for those on FRL (low income), for non-Special Education 
students compared to Special Education students, and for minority students compared 
to White students (except for Asian students in elementary reading). 



Almost all ALP subgroups had stronger high growth composites than WCPSS 
overall. 
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• Compared to WCPSS Level I and II students overall, ALP elementary students 
showed more positive high growth composites in reading at grades 4, 7, and 8, and 
math at grades 4, 5, and 7. However, 6th graders in ALP, similar to all WCPSS Level 
I and II students, did not show high growth in either reading or math. 

ALP Timing 

A regression analysis revealed that the timing of ALP sessions (during the day versus outside of 
the day or at both times) did not make a great deal of difference to individual students’ learning. 
However, comparisons of schools with the highest and lowest growth for Level I and II students 
revealed that schools with the highest growth for Level I and II students were more likely to 
offer assistance outside of the regular school day than the schools with the lowest growth. 

Improvements over Two Years 



Cohorts of 1998-99 3rd and 6th graders, followed over two years, revealed increasing 
percentages of students able to reach grade-level performance. 

• In reading, the percentage of 3rd graders in ALP who reached grade level rose steadily by 
grade 4 (32%) and grade 5 (59%). This increase was similar to middle school, where 
40% of ALP students reached grade level in grade 7 and 57% by grade 8. 

• In math, the percentage of students in ALP who reached grade level also increased, but 
the degree of improvement was greater in the first year than in the second. The 
percentage of 3rd graders in ALP who reached grade level rose to 59% at grade 4 and 
69% at grade 5. The percentage of 6'*’ graders in ALP who reached grade level rose to 
40% in grade 7 and remained at 40% in grade 8. 

Highest Growth Schools 

At both the elementary and middle school levels, schools with the highest growth for Level I and 
II students, compared to those with the lowest growth, were more likely to: 

• offer ALP sessions outside of the school day, 

• assist some low Level III students, and 

• have more teaching staff involved. 

Other successful strategies varied by level. At elementary, the most successful schools were 
more likely to: 

• vary strategies for ALP and the regular instructional day, 

• have strong parent communications, with specific ways for parents to help, 

• use fewer volunteers, 

• start ALP by October. 
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At the middle school level, successful schools were more likely to: 

• have higher ALP participation rates, 

• use fewer instructional strategies during ALP, 

• have a special period during the day for remediation/enrichment at least part of the year 
(special electives, extended advisory or team times), 

• have slightly fewer FRL students (15% vs. 21%), and Level I and II students (avg. 138 
vs. 164), but more English as a Second Language (ESL) students (4% vs. 1%), 

• have a year-round calendar. 

Based on telephone interviews, the elementary and middle schools with the best gains for Level I 
and II students also exhibited evidence of effective school characteristics, such as strong program 
staff, supportive principals, a focus on instruction, high expectations for students, and parent 
involvement. 



RECOMMENDATION 

Overall, evidence suggests ALP has had a positive impact on the ability of students who score 
below grade level to reach grade-level achievement. However, some schools were clearly more 
successful than others. Changes in participation rates and use of the allocations at schools also 
suggest ALP is continuing to evolve. For schools that are successful based on EOG and ABC 
results, this increased flexibility seems appropriate. However, for those schools who did not 
achieve high growth on ABCs for their Level I and II students, relevant central staff are 
encouraged to review use of ALP with school staff to see if improvements in recruitment or use 
of ALP dollars could make a positive difference in results. In addition, it may be appropriate to 
review ALP guidelines more formally in light of evaluation and achievement results to see if 
modifications are appropriate (with input from elementary and middle school staff). 
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Evaluation Plan 

E&R’s evaluation addresses the basic question of whether low-performing WCPSS students are 
progressing at a faster rate towards grade-level performance since the addition of ALP. This 
report summarizes results on: 

• The types of assistance available to students who show low achievement 

• Student eligibility and participation in ALP 

• The structure and implementation of ALP programs 

• Staff perceptions of the effectiveness of ALP at each level 

• The impact of ALP and other assistance on student achievement test results 

• The impact of timing of ALP services on the achievement gains of students 

• Schools that were most and least successful in promoting strong growth for Level I and II 
students and possible beneficial practices 

The full evaluation plan is available from the WCPSS Evaluation and Research Department. 



DATA SOURCES AND METHODS 

Data sources used are summarized in the next table. To the extent possible, information was 
obtained from central office data files and contacts (e.g., student demographics, service by 
various programs, and test performance). Schools were asked only about the site-specific ways 
in which they implemented their ALP program. 

Both qualitative and quantitative analyses were utilized. Many analyses were descriptive in 
nature. We used regression analyses to assess various aspects of the effectiveness of the ALP 
programs on student progress. More specific information on methods is provided in the relevant 
sections of the report. 
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Figure 1 

Data Sources for ALP Evaluation 3-8, 2001-02 



Data Source 


Description 


ALP Program Plan 


Fall descriptions of school programs provided to program 
director 


ALP Feedback Form 


Spring updates on school programs and feedback on 
effectiveness provided to E&R 


Middle School Comprehensive 
Student Roster 


Subject areas of service provided to all eligible students 


Data Capture Sheets 


K-5 literacy and math assessment profile status for students 
as well as ALP service information. 


Locators 


Demographic information on WCPSS students at specific 
points in time 


Masterbuild for End-of-Grade (EOG) 


EOG test scores matched with demographic information 


Program Records 


Documents and information from central program staff 
about program budgets, training, and implementation 


Interviews 


Interviews with key staff about program implementation 
and success (staff at schools and central office) 


ABCs Analyses 


Information on growth of students by school and level 
based on state ABCs regression analyses 
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Instructional Assistance Available to Low-Performing Students 



The 2001-02 school year was the third for the ALP 3-8 program. However, it is important to 
realize that other efforts were also in place to support students in K-8. A description of ALP and 
some other major efforts follow. Attachment 1 provides information on programs available by 
school. 



ALP PROGRAM OVERVIEW 

ALP History 



In 1998-99, WCPSS provided additional support funds to schools with a concentration of low 
income students and/or low achievement. Fund use was very flexible, with schools submitting 
plans for their programs to Instructional Services. In spring 1999, a system wide committee met 
to discuss what it would realistically take to accomplish Wake County Public School System’s 
Achievement Goal 2003, which is to have 95% of our students testing at or above grade level by 
2003, as measured by NC EOG testing at grades 3 and 8. National research, curriculum theory, 
and personal experiences were all considered in recommending that a new program, the 
Accelerated Learning Program (ALP), be instituted in 1 999-2000. Funds were allocated on a 
per-student basis, so that schools with the most students in need received the most funds. For 
1999-2000, schools were asked to develop implementation plans for ALP within the following 
more specific parameters: 

• ALP students were to be provided up to 22 extra days of instruction, based on individual 
needs. One initial parameter of ALP was that all instruction take place outside the 
regular school day, such as Saturdays, teacher workdays, holidays, student vacation time 
(during intersessions of year-round schools), before school and/or after school. Another 
parameter was that a variety of times be included in the school’s ALP plan, with no more 
than one third of the ALP “days” to be before or after school instruction. 

• A Personal Education Plan (PEP) was to be developed and monitored for each student, 
including objectives and strategies for meeting each student’s needs. 

• Teachers and volunteers who provided services to the ALP students were to be “highly 
trained.” 

• Parent involvement was to be encouraged in terms of student participation, supporting the 
learning process at home, communicating with teachers, supporting school staff, and 
attending parent/teacher conferences. 

• The community ALP was initiated, which encouraged individuals and groups (churches 
and other organizations) to volunteer in schools to support students. Volunteers were 
trained in literacy and/or math before working as tutors. 
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In 2000-01, ALP was expanded from grades 3-8 to K-12, and a summer academy was added for 
grades 3-8. Expansion was possible due to a one-time transfer of $7 million from capital 
improvement to operating expenses by the Wake County Commissioners in November 2000. 
Original guidelines were in place, except that: 

• The timing parameters were relaxed somewhat to permit ALP services to be offered 
during the school day. 

• The Challenged Schools and School Improvement Program Grant programs were added. 
Challenged Schools were those with the highest concentrations of low income students, 
and the additional funds could be used to extend existing services or serve groups 
typically not served (e.g., those who showed test scores just above the grade-level-cut 
scores). Schools that did not qualify for the Challenged Schools program could apply for 
a School Improvement Program Grant based on special needs at their campus. These 
schools had lower levels of need schoolwide but had special circumstances or needs. 



ALP 2001-02 



No major changes were made in the ALP 3-8 program for the 2001-02 school year. Schools 
were given even more flexibility, however, in terms of the time of day they used ALP funds. 
Allocations for 2001-02 were similar to 2000-01, except that allotments per student were based 
on a 12:1 pupihteacher ratio rather than 10:1 due to an increase in the number of challenged 
schools. ALP 3-8 served 7,285 students in 2001-02 at a cost of about $876 per student. The next 
table summarizes allocations for each component of the ALP 3-8 program. 

Figure 2 

ALP Program 3-8 Allocation Overview for 2001-02 



PROGRAM 


Budget 


Funding Basis 


Purpose 


ALP 3-5 


$2,725,326 


$ per student plus 

administrative 

costs 


Improve individual students’ reading and 
math skills 


ALP 6-8 


$2,429,016 


$ per student plus 

administrative 

costs 


Improve individual students’ reading and 
math skills 


ALP Challenged 
(K-8) TOTAL 

Elementary 

Middle 


$1,230,174 

($1,028,549) 

($201,625) 


Schools with 30% 
or more low- 
income students 
(eligible for FRL) 


Extra support to improve reading and math in 
schools with greater needs 

j 


|TOTAL ALP 3-8 


$6,384,516 
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OTHER ASSISTANCE AVAILABLE 

Title I 

Title I, a federal funding source, provides funds for school systems to assist schools that have 
high poverty concentrations. Within these schools, students with the lowest achievement and 
greatest needs, identified on multiple criteria, are identified for service. In 2001-02, most Title I 
schools opted to serve students in grades K-2 with the ALP K-2 literacy program (also known as 
ALP II). Local funds were also used for this program. A few schools continued to provide 
literacy assistance at grades 3-5, usually in addition to K-2. Title I students in grades 3-5 
received about 40 minutes per day four to five days per week. Service was provided during the 
regular school day. Most students were served across the full year, which results in about 100 
hours of service per child. Students with the greatest needs were served first, and teacher 
capacity did not always allow all students to be served even in the designated schools. 

Class Size Reduction fCSR) 

CSR provides federal funds to reduce class size in grade K-3 classrooms. Small classes have 
been found to impact student achievement, especially in the early grades. In 2001-02, WCPSS 
had 40 teachers assigned to 23 schools for this purpose. Most schools used the extra teachers to 
create a new class at a grade level (preferred); a few reduced class sizes for part of the day 
through a teaming approach. Thus, students benefited all day or part of the day based on the 
model chosen by their school. 

Project SOAR 

Project SOAR is an after-school enrichment program operating at seven WCPSS schools (Lead 
Mine Elementary, Leesville Middle, Athens High, Fuquay-Varina High, Sanderson High, 
Southeast Raleigh High, and Wake Forest-Rolesville High), and is made possible by a federal 
grant from the 21st Century Community Learning Center. Each individual WCPSS program is 
unique but has been designed to increase students’ resiliency utilizing four common factors; 
academic competence, a sense of belonging, a sense of usefulness, and personal potency. In 
2001-02 Project SOAR sites offered 8-15 hours of after-school programming per week, 
including homework assistance, computer use, recreation, and enrichment activities such as 
photography, Web design, art, etc. SOAR also offered parent education workshops, discussion, 
groups, and student-parent learning activities. 

Support Our Students (SOS) 

SOS is a statewide effort to establish high-quality after-school programs to promote academic, 
social, and character-building experiences for students. In Wake County, 4-H Youth 
Development has coordinated the program for ten years at two community-based sites and six 
middle schools. Students are scheduled to attend two to five times per week. SOS staff has had 
some training in the NC Standard Course of Study. 
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English as a Second Language (ESP 

Students with limited ability in English are provided additional support through ESL services at 
many WCPSS campuses. This program helps students make the transition to English while 
supporting them in their academic work. In 2001-02, ESL was available on 63 campuses (42 
elementary, 12 middle, and 9 high schools). 

Helping Hands 



The goal of the School/Community Helping Hands Project is to promote African-American 
males’ success in school and in their community. The focus is on improving students’ 
metacognitive skills, interpersonal skills, self-esteem, and leadership skills. To accomplish this, 
African-American male educators and community volunteers work with small groups and 
individual students. During the 2001-02 school year. Helping Hands served more than 350 
students. 

Communities in Schools (CIS) 

CIS connects community resources with students and families through the school site. CIS tries 
to prevent school failure through activities such as mentoring and tutoring. Some efforts are in 
schools while others are at community sites. They have also assisted with training for ALP 
volunteers for schools. 

Other 



In addition, some schools have additional tutoring or support services funded through small 
grants. Parent Teacher Associations, or in other ways. Some churches, community groups and 
private firms also offer tutoring and/or mentoring services. 
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Level I and II Achievement Trends Grades 3-8 



MOVEMENT UP AND DOWN ACROSS ACHIEVEMENT LEVELS 

Achievement of Goal 2003 requires nearly all students currently scoring in Levels I and II to 
reach grade level based on EOG scores. For students scoring close to the cutoff scores between 
the two levels, some movement is to be expected based on measurement error alone. However, 
our effectiveness in reaching Goal 2003 is diminished to the extent that students drop back from 
Level III or IV achievement to Level I or II. With successful support, the number of students 
moving up to grade level should exceed those going down. 

Level Scores 



Most WCPSS students (about 89%) scored at Level III or IV both in spring 2001 and 2002. 
However, some students moved up from Level I or II to III or IV each year, while others moved 
down from Level III or IV to Level I or II. Our overall success in reaching Goal 2003 is 
influenced not only by the percentage of Level I or II students able to increase their scores to 
Levels III or IV, but they those who drop from Level III or IV to Level I or II. For students 
scoring close to the cut scores between the two levels, some movement is to be expected based 
on measurement error alone. (However, with successful support, we hope the number of 
students moving up to grade level exceeds those going down). As the next graph illustrates, 
more students moved up to Level III or IV than down to Level I or II the desired pattern. By 
subject: 



• In reading, 7.1% of the students moved up to Level III or IV, while 3.9% moved down to 
Level I or II (similar to 2001 and an improvement over 2000). 

• In math, 5.6% of the students moved up to Level III or IV, while 3.1% moved down to 
Level I or II (an improvement over last year resulting from fewer students moving down 
a level). 

• The net increase in students in Level III or IV was 1,358 students in reading and 1,056 in 
math. 
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Figure 3 

Patterns of Achievement Level Performance Between Spring 2001 and 2002 
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Movement Up to Grade Level 

If we focus just on those students who scored in Level I or II each spring, the percentage of Level 
I or II students able to reach grade level within one year has been steadily increasing each year 
since support such as ALP was added. As illustrated in the next graph, only 36% of students 
were able to reach grade level between spring 1997 and 1998 (before support for ALP Regular or 
Challenged Schools began), while more than 50% were able to reach grade level between spring 
2001 and 2002 (with new ALP supports in place). ALP expanded the system’s capacity for 
service so that all students at grade, 3-8 scoring below grade level could be offered assistance. 

(In 1998-99, grants and locally funded programs were able to serve only 54% of the low 
achieving students.) 
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Figure 4 

Percentage of Students Who Scored in Level I or II the Prior Year 
and Moved Up to Grade Level the Next Year: Grades 3-8 
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Between spring 2001 and 2002, by grade: 
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• In reading, the highest percentage of students reached grade level in grades 3 and 8 (over 
50%). Grade 6 stands out as having the least success (22%). 

• In math, the highest percentage of students reached grade level in grade 4 (68%). Grade 
3 stands out as having the least success (28%). 
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Net Increases 

The following charts illustrate positive trends in students’ score movement up and down across 
grade levels over time: 

• In reading, the patterns improved considerably between spring 1999 and 2000 and spring 
2000 and 2001 and stayed about the same from spring 2001 to 2002. 

• In math, there was a steady improvement in patterns across the three years. 

• Overall, net increases were still greater in reading than math. 

Figure 5 

Net Increase in Students at Levels III and IV on EOG Grades 3-8 

Reading 




Math 
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As shown below, in reading, the net increase was more positive at grades 3-5 than at grades 6-8. 
In math, the opposite was true, with a more positive net increase at grades 6-8 than at grades 3-5. 



Figure 6 

Net Increase in Students in Level III or IV by Grade Span Spring 2001 to 2002 
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Level Declines for Level III Students 



Students who score just above the scale score cut points on EOG are of special interest in terms 
of meeting Goal 2003. A closer look at level declines among Level III students reveals that, as 
expected, the percentage of students dropping from Level 111 or IV to Level 1 or 11 was higher for 
those who scored close to the scale score cut point (within two points) than for those who scored 
further above the cut point. This was true in both reading and math. (See Figure 7.) 

A second question of interest is whether students who scored low in Level III (within two and 
four scale score points) were supported through ALP. That question is addressed in the ALP 3-5 
and 6-8 sections of this report. 
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Figure 7 

Students Dropping from Level III in Spring 2001 to Level I or II in Spring 2002 by Their 
Proximity to the Level II-III Scale Score Cut Point in Spring 2001 




Number Scale Score 
Points Above Cut in 
Spring 2001 


Grades 3-5 


Reading 


Matb 




# 


% 


# 


% 


1-2 points 


187 of 792 


23.6 


254 of 1,170 


21.7 


3-4 points 


242 of 1,358 


17.8 


192 of 1,451 


13.2 


5 or more points 


318 of 5, 625 


5.6 


211 of 4,952 


4.2 


Total 


747 of 7,775 


9.6 


657 of 7,573 


8.6 




Grades 6-8 


1-2 points 


280 of 874 


32.0 


287 of 862 


33.2 


3-4 points 


217of 1,135 


19.1 


171 of947 


18.0 


5 or more points 


335 of 4,973 


6.7 


223 of3,981 


5.6 


Grand Total 


832 of 6,982 


11.9 


681 of 5,790 


1 1.7 



Note: To be included in the analysis, students had to have scores from 2001 and 2002. Only non-retained students 
are included. 



High growth based on the state’s ABC regression formulas will help us reach grade level with 
students who score below grade level. 



ABC Scale Score Growth 

State ABC regression analyses reveal that: 

• At both the elementary and middle school levels, WCPSS showed high growth for 
students who initially scored in Levels I or II the last two years. This is a positive 
pattern for Level I or II students, which should allow most to reach grade-level 
performance over time. 

• Growth was stronger for Level I or II students than for III or IV students at both the 
elementary and middle school levels. This is desirable for accomplishing Goal 2003. 
However, high growth for all groups is desirable; therefore, instruction for Level IV 
students at the elementary level and Level III and IV students at the middle school level 
may need some adjustments. 

• At both the elementary and middle school grades, growth was stronger for students in 
Levels I or II than for low income (FRL), Black, and Special Education students. Our 
students in Level I or II who fit in more than one of these categories represent the greatest 
challenge for WCPSS educators in reaching the WCPSS Goal 2003 as well as the No 
Child Left Behind (NCLB) federal requirements. 
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Figure 8 

ABC High (Exemplary) Growth for Key Subgroups in Spring 2001 and 2002 
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2001 


2002 1 


1 Group 


Grades 3-5 


Grades 6-8 


Grades 3-5 


1 Grades 6-8 


1 Systemwide (All) 


.10 


-.04 


.02 


1 .04 


Levels 1 and 11 


.77 


.24 


.80 


1 


Level 111 


.10 


-.19 


.04 


-.25 


Level IV 


-.22 


-.03 


-.30 


.22 


Low Income 


-.20 


-.66 


-.28 


-.52 


Not Low Income 


.19 


.08 


.10 


.17 


Special Education 






-.07 


-.59 I 


1 LEP 


. ■ . 




-.07 


.46 1 


Asian 




.r .. _ • , 


.29 


!64 1 


Black 






-.24 


-.43 1 


Black Males 


-.26 


-.57 


-.21 


-.54 ] 


I Black Females 


-.16 


-.48 


-.26 


-.34 1 


Hispanic/Latino 






.13 


-.18 


Native American 






.03 


.71 


White 






.10 


• 19 


White Males 


.20 


.05 


.05 


.10 


White Females 


.21 


.16 


.16 


.29 


Multiracial 






-.15 


.11 1 




Notes: Bold=ABC High Growth Standard was met. 

Shading means data was not available for 2000-01 ; breakdowns were new in 2001-02. 

LEP = Limited English Proficiency 

Source: Disaggregated ABC Composites as of Oct. 2002. 



By grade, among students who initially scored at Levels I or II; 



• The strongest growth in reading was evident at grades 5, 7, and 8. Grades 3 and 6 
showed the greatest need for improvement relative to ABC high growth standards. 

• The strongest growth in math was evident at grades 4, 5, and 7, with grades 3, 6, and 8 
showing need for improvement to reach exemplary growth standards. 




Thus, grades 3 and 6 showed the most need for improvement. A task force has explored possible 
reasons and solutions for grade 6 performance and attempts are being made to improve the 
middle school transition. At grade 3, possible reasons could relate to instruction or assessment 
issues. The fall pretest at 3rd grade is much shorter and less reliable than the EOG test given at 
the other grades. Nevertheless, grade 3 is critical given that Goal 2003 targets this grade, so 
instructional adjustments should also be considered. 
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Growth by School 

The majority of WCPSS schools showed high growth for Level I or II students, with more 
consistent results at the elementary than middle school level. 

• 84% of our elementary (63 of 75) schools showed high growth for Level I or II students. 
Nine showed expected growth, and three did not show expected growth for their students 
scoring in Level I or II. 

• 56% of our middle schools (14 of 25) showed high growth for Level I or II students. 

Four showed expected growth, and five did not show expected growth for Level I or II 
students. 

The achievement trends part of the ALP 3-5 and ALP 6-8 sections of this report include 
descriptions of practices common to the schools achieving the highest growth for Level I or II 
students. 
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ALP 3-5 Programs 



PARTICIPATION 



Eli£ibilitv Criteria 

Allocations for ALP were based on counts of students with test scores (below grade level) the 
previous spring. However, other struggling students could be served as space allowed (e.g., 
students who had scored just above the cut score for Level III). 



• At grade 3, allocations were based on students who showed WCPSS literacy profile 
scores below a book level of 23-24 or math profile scores below grade level in two or 
more of the four math strands. 

• At grades 4-8, allocations were based on students scoring below grade level in reading 
and math on EOG in spring 2001, plus Special Education students not tested on the 
standard form of EOG (computer adaptive, checklist, or portfolio). 



Participation 




Data on ALP participation was collected in a different way in 2001-02 at grades 3-5. This 
chemge in data collection method appears to have impacted reporting somewhat, and 
comparisons across years must be made cautiously. Previously, the ALP coordinator was in 
charge of the data collection, and classroom teachers were less involved in many schools. To 
reduce paperwork, the Data Capture form was revised to include student participation in ALP as 
well as K-5 assessment data. Classroom teachers completed the form. We suspect some over- 
and under-reporting resulted. 



• Over-reporting: We suspect some students marked as served in ALP were actually 
receiving service through other efforts (which may not have been dependent on low 
achievement). One reason was that the number of students scoring Level III or IV in both 
reading and math who were reported as being in ALP rose considerably, from 502 in 2000-01 
to 1,390 for 2001-02. Schools with the highest number of Level III or IV students reported 
as participating in ALP were called to verify their participation. In nearly all cases, the 
students were not in ALP, but rather were in an enrichment or other support effort at the 
school. To address this issue, we re-coded all students who scored in Level IV in both 
reading and math from ALP to “not ALP”. These students have been excluded from further 
counts. (We were less sure about Level Ills and therefore made no changes to students’ 
status.) 




Under-reporting: Sometimes schools used ALP funds to support a school initiative, but the 
program was not called ALP. In this case, teachers may not have reported a student as being 
in ALP who actually was in a program funded at least partly through ALP. In addition, if 
schools used ALP funds for an ALP teacher, or part of an ALP teacher, to help students 
during the day, the classroom teachers who completed the Data Capture form may not have 
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realized that these students should be marked as being in ALP. We had no way to catch 
such reporting errors. 

Thus, the basic issue is that classroom teachers were not always aware of the source of funding 
of certain efforts. One idea for 2002-03 is to ask principals or ALP contacts to communicate 
how ALP funds were used to their faculty at the time of the data collection. 

Because of the differences in data collection and reporting, we will compare participation rates 
only generally between 2000-01 and 2001-02. If we count as eligible those in initial allocations 
plus those with higher test scores served based on teacher recommendations both years, the 
participation rate for 2000-01 was 77%, compared to 71% for 2001-02. Thus, participation 
appeared to be down slightly. However, the decline is actually slightly greater for those initially 
considered eligible (primarily students with low test scores), because of the rise in Level III 
students served. Overall, 1,654 students with no Level I or II scores (including some newly 
enrolled students who had no EOG scores) were served in ALP in 2001-02 compared to 502 in 
2000-01. Overall, 62% of Level I or II students who were eligible participated in ALP. (See 
Attachment 3C.) 

The number of students at or above grade level served in ALP was higher at grade 3 than at 
grades 4 and 5. This is likely because the grade 3 EOG pretest identified additional students as 
scoring below grade level. 

Participation by school varied from 28.6% to 90.3% of those eligible. (See Attachment 2A.) 



Figure 9 

Student Participation in ALP 3-5 2001-02 



Grade 


Below Grade 
Level* 


At or Above 
Grade Level 


Total Participants 


3 


1,132 


763 


1,895 


4 


1,091 


438 


1,529 


5 


949 


453 


1,402 


Total 


3,172 


1,654 


4,826 



Source: Data Capture Sheets May 2002 



Note: “Below Grade Level” for grade 3 was determined by grade 2 assessment, with grades 4-5 

determined by EOG reading and math levels from spring 2001. Special education students 
without regular EOG scores were also considered to have needs for allocation purposes. 
These represent students counted in allocations. Any student who had a Level I or II score in 
either reading or math (or both) is included in “below grade level”. Students in “At or 
Above” had no Level I or II scores. 



The following figure provides information on the characteristics of ALP students. Compared to 
WCPSS populations overall. Black and low-income students were significantly overrepresented 
in those eligible for and participating in the program, a trend that has been found over time. 
Achievement results on EOG tests show more minorities than non-minorities scoring in Level I 
or II (see Attachment 3A). 
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Of ALP students in grades 3-5: 



• 55.9% were Black, with 44. 1% comprising all other ethnic backgrounds, 

• 4.3% were receiving assistance through ESL programs, 

• 5 1 .6% were male and 48.4% were female, 

• 28.9% received Special Education services, and 

• 52.2% were low income (based on FRL eligibility). 



Figure 10 

Comparative Characteristics of Students in Overall WCPSS 3-5 Population 
and in ALP 3-5 Population, 2001-02 

100 % - 



I 




■ This Group's % of Total 3-5 60.1% 27.3% 5.9% ■ 4.2% 0.3% 2.3% 3.0% 25.7% 74.3% 16.3%;83.7%j 
WCPSS Population i ' ' ! I 



□ This Group's % of ALP 3-5 30.4% 55.9% 9.1% 1.9% 0.2% 2.4% : 4.3% 52.2% '47.8%'28.9%|71.1%' 
Participants ' ^ ; 



Participation as of May 2002. Percentage of population is from 20th-day counts, fall 2001. 



Other services available to students in grades 3-5 included: 







Title I (limited service at grades 3-5) 

Special Education 
ESL 

Communities in Schools (CIS)/Community Learning Partners (CLP) 
Project SOAR 
Helping Hands 

Partnership for Educational Success 
Parent and other volunteer tutors 
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• Small-group and individual instruction provided by special area teachers, the 
Instructional Resource Teachers (IRTs), the literacy teachers, and/or teacher assistants 

• School-based assistance efforts 

Specific programs available at each school are included in Attachment 1 of this report. 

ALP Support for Level III Students 

Examining EOG scores by subject, we checked to see whether students scoring low in Level III 
were receiving ALP support. Many school administrators have argued this support could prevent 
students from falling back to Level I or II. Because this analysis is by subject, counts of Level 
III or IV students served will not match those under participation. Some students included here 
scored a Level I or II in one subject and a Level III in the other subject. In the participation 
section, the Level III or IV students served had no Level I or II scores. Some school ALP 
programs serve students in both subjects regardless of test scores. 

Elementary Reading 

• At the elementary level, 2,150 students scored within four points of the lowest scale score 
cut point for Level II/III. About 33% of these students (706) received ALP support based 
on teacher reports. 

• Support was much more common for those who scored within two points of the cut 
(39.5%) than for those scoring three or four points above the cut (28.9%) or five or more 
points above the cut score (12.8%). 

• The percentage of elementary Level III students supported through ALP was much higher 
than at the middle school level. (This difference may have been exaggerated because of 
differences in data collection methods.) 

Elementary Math 

• Overall, 2,621 students scored within four points of the lowest scale score cut point for 
Level III. About 36% of these students (939) received ALP support based on teacher 
reports. 

• As with reading, support was more common for those who scored within two points of 
the cut (40%) than for those scoring three or four points above the cut (32%) or higher 
( 16 %). 

• Support was much more likely to occur at the elementary than at the middle school level 
for these students (although the difference is not quite as great as for reading). (This 
difference may have been exaggerated because of differences in data collection methods.) 



27 

o 

ERIC 



18 



Report No. 02.34 



Figure 11 

2001-02 ALP Service for Grade 3-5 Level III Students 
Based on Spring 2001 EOG Test Scores 



READING 


# Points Above Level II-III Cut Point 




1-2 pts 


3-4 pts 


5+ pts 


Total 


# 


479 


965 


4,905 


6,349 


NO ALP 






% 


60.48 


71.06 


87.20 


100 


# 


313 


393 


720 


1,426 


IN ALP 








% 


39.52 


28.94 


12.80 


100 


Total 


792 


1,358 


5,625 


7,775 



MATH 


# Points Above Level II-III Cut Point 




1-2 pts 


3-4 pts 


5+ pts 


Total 




698 


984 


4,139 


5,821 


NO ALP 






% 


59.66 


67.82 


83.58 


100 


# 


472 


467 


813 


1,752 


IN ALP 








% 


40.34 


32.18 


16.42 


100 


Total 


1,170 


1,451 


4,952 


7,573 



IMPLEMENTATION 

Each school’s ALP services varied within the guidelines for the program in terms of their ALP 
calendars (Saturdays, full days, half days, intersession days, before/after-school hours, etc.), 
approaches used, subjects emphasized, and staffing provided for the ALP sessions. The majority 
of elementary schools offered assistance in a combination of session times on school days and 
non-school days. Overall, 46.6% (34 out of 73) of responding elementary schools offered ALP 
for only one session time. 

Session Times Offered 



As shown in the following figure, after-school, Saturday, and during-the-day sessions were the 
most popular options used in elementary schools. 

Using ALP funds to provide students with additional support during the day has become more 
common over time in traditional-calendar schools. 

• In 1999-2000, ALP’s initial year, sessions were offered exclusively at various times 
outside of the regular school day. 
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• In 2000-01, these restrictions were relaxed, and after-school remediation and remediation 
during the school day became more common. One third of the schools offered some 
assistance during the day; most combined this with assistance at other times outside of 
the school day. 

• In 2001-02, 42.5% of the schools that reported hours offered some assistance through 
ALP during the school day, with 21.9% offering ALP exclusively during the day. 

Figure 12 

Session Times Offered for ALP 3-5 Programs Overall, 2000-01 and 2001-02 



All Schools 


2000-01 


2001-02 


Schools 


# Schools 


% Schools 


# Schools 


% Schools 


School Days 


Before School 


9 


12% 


4 


6% 


After School 


53 


70% 


38 


52% 


During the School Day 


23 


30% 


31 


43% 


Non-School Days 


Saturday 


46 


61% 1 


31 


43% ' 


Teacher Workday 


17 


22% 


9 


12% 


Intersession Days 


11 


14% 1 


9 


12% 



Source: Fall Program Descriptions and Spring ALP Feedback Form 
Return Rates; 73 of 78 (93.6%) schools reporting. 



Number of Hours Offered 



The number of ALP instructional hours each school provided varied considerably. 

• Based on 73 of 78 elementary schools reporting, a total of 1 1 ,771 hours were earmarked 
for ALP sessions, with an average of 161 hours per school (equivalent to 27 six-hour 
instructional days). However, because of inconsistencies in the interpretation of a 
question about the number of during-the-school-day hours offered by each school versus 
the number of during-the-school-day hours any individual student could have accessed, 
this is probably more hours than were actually available to individual students. In 
addition, during-the-day hours do not actually extend students’ learning time but modify 
a portion of it. Without taking during-the-day hours into account, schools offered 5,762 
hours of ALP - an average of 79 hours per school, or 13 six-hour instructional days. 
Saturdays and intersession days needed to last three hours to count as a full day for ALP. 
Reported hours ranged from 31 to 656. The median number of hours was 90. 
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• The average number of ALP hours (341) offered at year-round elementary schools for 
each track was substantially higher than at traditional-calendar elementary schools (136). 
However, it should be noted that summer-school sessions for traditional-calendar schools 
were not figured into the reporting because they had not yet taken place. 

Figure 13 

Number of ALP 3-5 Additional Hours Offered Overall and by Calendar Type 





# Schools 
Reporting 


Average 
Hours per 
School 


# Hours 
Reported 


All Elementary 


73 of 78 


161 


11,771 


T raditional-Calendar 


64 of 68 


136 


8,699 


Year-Round 


9 of 9 ■ 


341 


3,072 



Source: Program Descriptions 



Number of Hours Offered by Time Slot 

In terms of actual hours of assistance by time of the offering: 

• During-the-day sessions (6,009 hours) represented the highest proportion (51.1%) of the 
total number of hours offered. 

• After-school sessions (2,046 hours, or 1 7.4% of the total hours) were the next most 
popular. 

• Intersession (1 ,847, or 1 3.2%) and Saturday (1 ,340, or 1 1 .4% of total hours) times also 
were fairly common. 

• Before-school sessions accounted for only 2.1% of the total number of hours. 

• Teacher workdays, early release days, holidays, and all other hours combined accounting 
for 2.4% of the total hours. 



In 1999-2000, year-round schools offered ALP services almost exclusively during the 
intersessions. Because each year-round school offered sessions for four tracks during the school 
year, total number of hours was greater than for traditional-calendar schools, but the hours 
available to each student actually were fairly comparable to those offered by traditional-calendar 
schools. In 2000-01, 67% of year-round schools supplemented their ALP intersession services 
with sessions at other times. In 2001-02, 77.8% of schools offered ALP during other times as 
well as during the intersessions. Year-round calendar schools offered a lot fewer ALP hours 
during the day than traditional-calendar schools. 
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Figure 14 

ALP 3-5 Percent of Hours Offered at Each Time Slot 



100% I I ■2001-02! 

! !□ 2000 - 01 ' 

75% 

i 



50% -■ 



25% 




U/o — 

1 

1 


During the 
day 


j After school 


1 Intersession 


Sat. 


j Teacher ' 
! workday 1 


Before 1 

1 

school 1 


■ 2001-02 : 


51.1% 


! 17.4% 


i 15.7% 


1 1 .4% 


1 2.4% 1 


2.1% i 


;□ 2000-01 1 


14.0% 


' 38.0% 


12.0% ‘ 

1 


22.0% . 


i 7.0% ’ 


7.0% 



Note: 73 schools reported hours. Based on 1 1,771 hours. 
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Figure 15 

Comparison of Year-Round and Traditional-Calendar Elementary Schools 
by Percentage of ALP 3-5 Total Hours per Session Time 



Year-Round Elementary Schools 



Traditional-Calendar Elementary Schools 



After school 
7.1% 



Saturday 

2.9% 




Before Teacher 

school- -workday 
2 . 8 % 0 . 4 % 

Saturday 




During the 
day 
59.3% 



n = 9 n = 63 

Note: 9 year-round and 64 traditional-calendar schools reported hours. 



ALP Provided During the School Day 

The 1 6 elementary schools offering ALP services only during the school day reported providing 
a total of 3,993 ALP hours over the 2001-02 school year, with an average of 250 hours per 
school, or 41.6 six-hour instructional days. The schools provided ALP for an average of 29 
weeks of the school year. 

What students missed varied, though the most frequently reported activities missed were social 
studies (from which 50% of schools reported pulling students at least part of the time) and 
science (37.5%). Other missed subjects that two or more of the 16 schools reported were 
computer lab. Drop Everything and Read (DEAR), language arts, reading, recess, and/or 
specials. The subjects students missed varied in nine schools and remained the same in eight. 

Fourteen of the schools reported using their ALP programs to give students extra time in the 
subject needed, and one school offered small-group instruction in the same amount of time in the 
subject as other students; one school combined approaches. All schools pulled students from the 
classroom to provide ALP, and three schools also offered team-teaching as well. One school 
utilized both pullout and advisory methods. 
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Instructional Approaches 

Elementary schools used several basic instructional approaches in ALP: targeted instruction, 
team teaching, smaller classes throughout the day, electives, advisory sessions, and independent 
small-group instruction in the classroom. Subject areas addressed were reading, mathematics, 
and writing. Tutoring focused specifically on individual student needs. Targeted instruction 
featured small-group tutoring on specific skills within a subject area. Electives provided general 
help in a subject area, either individually or in small groups. The approach used for each student 
was based originally upon the student’s assessment data and subsequent Personal Education Plan 
(PEP). 

More than one third of the elementary schools (36%) provided some combination of these 
approaches for all three subjects. Targeted instruction (tutoring) was used most often (by 85% 
of schools reporting), followed by team teaching (24%) and independent small-group instruction 
(18%). Electives, advisory sessions, and/or smaller classes throughout the day were used by a 
total of 21% of schools. 

Schools were also asked to identify whether their version of ALP offered students extra time in 
the subject(s) in which they needed additional help, or small-group instruction for the same 
amount of time as other students. Two thirds of schools (64%) reported offering extra time in 
the subject, 1 5% of schools offered the same amount of time in smaller groups, and 21% of 
schools provided both extra time and small-group instruction. 

Staffin£ 

ALP guidelines recommended using highly trained certified teachers from the home school to 
conduct the ALP sessions. Schools staffed their version of ALP with 1 to 17 staff members 
(excluding volunteers). Schools reported fewer difficulties in recruiting staff in 2001-02 than in 
2000-01. As shown in the next figure: 

• Most schools (91%) reported using some of their own teachers for ALP. Overall, 
schools’ own teachers represented about 78% of all staff who were teachers for ALP. 

• More than half (58%) of elementary schools reported using other staffing assistance as 
well as their own teachers. 

• About 30% of the reporting schools used volunteers as part of ALP. This ranged from 0- 
18 volunteers per school. Schools that utilized them had an average school use, 
therefore, of 5.3 volunteers; five schools reported 10 or more volunteers. 
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Figure 16 

ALP 3-5 Staffing by Type: 2000-01 and 2001-02 
Percent of School Using Various Types of Staff 
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I 

30.3% ' 



Note: 74 schools reported in 2000-01, 76 schools reported in 2001-02 
Source: Spring ALP Feedback Form 



Parent Cooperation 

Most schools (61 .8%) reported that parent cooperation with their version of ALP was very high; 
28.9% reported that parents were somewhat cooperative. Only a handful of schools said that 
parents were minimally supportive (9.2%) or not supportive at all (1.3%). 

Successes and Challenges 

Based on the ALP Feedback Forms, elementary schools considered the greatest success of ALP 
to be children’s learning (76.6%) followed by staff commitment (72.7%) and children’s 
enthusiasm (48.1%). Schools found that student motivation (36.4%) and individualizing 
instruction (29.9%) were the most challenging aspects of their ALP programs. Compared to 
2000-01, student learning was seen as the biggest success more often. Recruiting staff and staff 
burnout were seen as challenges less often, but in 2000-01 and 2001-02 student motivation was 
more often seen as a big challenge. 
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Figure 17 

Successes and Challenges in ALP 3-5 



1 “What Was the Biggest Success with ALP?” 


Success Described 


% Schools 


% Schools 




2000-01 


2001-02 


Students’ learning 


56.4% 


76.6% 


Staff commitment 


65.4% 


1 72.7% 


Student enthusiasm 


56.4% 


48.1% 


Attendance 


43.6% 


44.2% 


Parent cooperation 


25.6% 


24.7% 


Transportation 


20.5% 


18.2% 


Other 


5.1% 


6.5% 


Parent involvement 


7.7% 


2.6% 


Challenge Described 




Student motivation 


20.5% 


36.4% 


Individualizing instruction 


20.5% 


29.9% 


Recruiting staff 


41.0% 


27.3% 


Student attendance 


23.1% 


26.0% 


Student behavior 


20.5% 


26.0% 


Other 


11.5% 


14.3% 


Staff burnout 


34.6% 


13.0% 


Transportation 


29.5% 


10.4% 


Student learning 


5.1% 


6.5% 



Source: ALP Feedback Forms 



ALP 3-5 STUDENT ACHIEVEMENT TRENDS 

Basic trends for the achievement of Level I or II students are described in the section entitled, 
Level I and II Achievement Trends Grades 3-8. This section will focus on analyses specific to 
ALP participants. 

ALP Effectiveness Based on ABC Models 

One analysis we conducted on ALP effectiveness utilized the state ABC expectations for high 
growth for Level I or II students as an external standard. High growth on the state ABC 
regression model brings students to grade level or closer to it. Using the regression programs 
used by the state in ABC Tools, we treated all students in ALP for reading as if they were one 
school. We then ran the same type of analysis for all those served in math through ALP. 
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As shown in the following graphs, ALP students in grades 3-5 overall showed high growth in 
both reading and math. Performance by subgroup was also strong. 

• All subgroups met the high growth standard in reading and all groups except LEP 
students met the high growth standard in math (LEP students met expected growth 
standards). 

• Growth was relatively stronger for students not on FRL than for those on FRL (our 
indicator of low income) and for non-Special Education students compared to Special 
Education students. 

• Black students showed relatively lower growth than students of other racial groups. 
Specific values and group sizes are shown in Attachment 4. 



High Growth Composite High Growth Composite 
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Figure 18 

ALP Grades 3-5 High Growth Composite by Group 2001-02 

READING 




Note 1 : Bars are not shown for groups of less than 1 0. For groups of 1 0 or more, group size ranged from 
39 for Asian to 1,355 for Black. LEP students=78. All students =2,226. 

Note 2: LEP= Limited English Proficiency F/R= Free or Reduced-Price Lunch 



MATH 




Note 1: Bars are not shown for groups of less than 10. The smallest groups shown are Asian (18) and LEP 
(40). All students = 1,308. 

Note 2: LEP= Limited English Proficiency F/R= Free or Reduced-Price Lunch 



By grade, elementary students in ALP met ABC high growth standards in all grades in reading 
and math. Compared to all Level I or II students in WCPSS, high growth composites were 
similar in reading and higher in math (remember these groups overlap). Compared to WCPSS 
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overall, results were more positive for ALP students by subgroup. In general, results are positive 
and point out the benefit of making assistance available to all those in need (which was not 
possible before ALP). 



Figure 19 

ABC High Growth Composites (High GC) by Grade 2001-02 Grades 3-5: 
ALP, All Level I or II, and All WCPSS Students 



Group 


1 Grade 3 


Grade 4 


Grade 5 


All 


High 

GC 


# 

Students 


High 

GC 


# 

Students 


High 

GC 


# 

Students 


High 

GC 


# 

Students 








■ Reading ;..v. 










ALP 


.14 


983 


.28 


610 


1.95 


633 


.77 


2,226 


All Level I-II 


.39 


1,404 


-.02 


1,225 


1.95 


940 


1 NA 


3,569 


All WCPSS 


-.4 


7,365 


-.51 


6,936 


.55 


7,027 


NA 


21,328 








'i. '"■I"' - ' ■■■'■ 


Math ;,v 










ALP 


.04 


454 


2.28 


584 


.99 


270 


.81 


1,308 


All Level I-II 


-.04 


1,404 


2.16 


1,225 


.84 


940 


NA 


3,569 


All WCPSS 


-.44 


7,365 


.81 


6,936 


.15 


7,027. 


NA 


21,328 



Note 1: BOLD means group met ABC exemplary growth standard 

Note 2: At district level, high growth composite is calculated across subjects. 

Note 3: NA= Not Applicable. 



Highest Growth Schools 



Overall, 84% of WCPSS elementary schools showed exemplary growth for Level I or II students 
in 2001-02. This shows that strong growth was widespread for our lowest students. We 
contrasted the characteristics of the 10 schools with the highest growth for Level I or II students 
(see the figure below) with those of the schools with the lowest growth. It is important to 
recognize that this is primarily a comparison between excellent schools and good ones (not poor 
ones). We compared both characteristics of the student bodies as well as the ALP programs 
based on school profiles, ABC results, and ALP Feedback Forms. We also conducted telephone 
interviews with the schools that achieved the highest growth with Level I or II students. 

Program descriptions for the highest growth schools are included in Attachment 5A. 
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Figure 20 

Ten Highest Growth WCPSS Elementary Schools for Level I or II Students: 
ABC Performance and Exemplary Growth Composites 



Elementary 

School 


Leve 

Performance 

Composite 


II/II 1 

High Growth 
Composite 


% Eligible 
Students 
Participating 


% 

Free/Reduced- 
Price Lunch 


% ESL 


Baucom 


83.6 


2.9 


61.0% 


10% 


” o % 


Apex 


84.7 


2.4 


40.0% 


14% 


0% 


Lockhart 


83.7 


2.2 


76.8% 


46% 


18.1% 


North Ridge 


80.8 


2.1 


77.5% 


31% 


8.2% 


Wiley 


73.3 


2.1 


60.0% 


32% 


19.2% 


Weatherstone 


79.5 


2.1 


43.0% 


20% 


8.8% 


Vance 


69.3 


2.0 


65.0% 


44% 


8.4% 


Wendell 


74.0 


1.8 


72.0% 


46% 


0% 


Cary 


71.9 


1.8 


47.0%* 


34% 


9.5% 


Middle Creek 


67.7 


1.7 


69.0% 


33% 


0% 


Range/ Avg. Top 
Schools 


68.5 


1.7 to 2.9 


55% 


30% 


6.4% 


Range/Avg. 
Lowest Schools 


52.2 


-.10 to .98 


64% 


30% 


4.0% 



♦ Under-reporting suspected 



In comparing the elementary schools with the highest and lowest growth, we found both sets of 
schools had similar percentages of low income students (FRL average about 30%), and numbers 
of LI-II students (125 vs. 122). The top schools actually had a slightly lower percentage of 
students participating in ALP of those eligible (55% vs. 64%) and slightly more ESL students 
(6% vs. 4%) than the lowest schools. 

Successes and challenges in ALP were fairly similar for schools with the highest and lowest 
growth composites for Level I or II students. Most strategies named by the schools with the 
highest and lowest growth were the same, with at least 70% of both groups of elementary 
schools reporting use of. 

• Math manipulatives 

• Frequent feedback to students 

• Leveled books 

• Smaller groups at key times 

• Individualized instruction 

• Supplemental materials 

• Curriculum mapping 

• High-interest reading materials 
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The schools with the top gains were more likely to: 



• Vary strategies used in ALP and regular classes to support Level I or II students 

• Use learning games as part of ALP 

• Provide ALP outside of the regular school day (only one offered ALP exclusively during 
the day compared to four of the low schools) 

• Have strong parent communication, including specific ways to support their child’s 
success 

• Have more teaching staff involved in ALP (8 vs. 4) 

• Have smaller numbers of volunteers for ALP 

• Assist some low Level III students 

• Start ALP early (by October) 



Phone interviews with administrators of the elementary schools with the highest gains for Level I 
or II students provided additional insights about the nature of their schools. Strong evidence of 
many characteristics of effective schools existed, with some additional themes. 



• Instructional leadership: Many principals of these schools indicated that they chose strong 
staff and worked collaboratively with them, facilitating the ALP program but trusting staff 
judgment on best practices. Administrators set the tone for high expectations for student 
success and had a strong focus on instruction. 







Student focus: School staff were aware of all students’ needs, had high expectations for all 
students, and offered frequent assessment and monitoring of student success. Most involved 
low Level Ills in ALP. 



• Instruction: Strong schools tended to offer support during regular instruction hours as well as 
in ALP programs outside of the school day. Other tutoring programs also were available. 

All chose some materials they found particularly helpful and used them consistently. Some 
ALP programs were more structured than others. Many served students in both reading and 
math. 

• Staff: Principals noted that staff were highly committed to student success and had high 
expectations. They reported no problems recruiting staff for ALP and often had more staff 
volunteers than were needed. Staff were characterized as being committed and experienced. 
Many administrators said that their ALP staff requested returning to the program from year to 
year. 



• Parent involvement: ALP programs offered parents specific instructions on how to help their 
students. Communication with parents was characterized as strong, with joint responsibility 
for student success. Some schools sent materials home with students for extra practice. 







School climate: The climate at these schools was definitely not stress free, but appeared to be 
highly focused on meeting Goal 2003. Staff relations with one another seemed positive, and 
reflecting of mutual trust and collaboration. 
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Timing of ALP Help; 3 -5 




The analysis of ALP timing examined whether or not the time of ALP service affected growth 
across grades 3-5. Only students who were eligible for ALP and participated in ALP were 
examined. In the analysis, students attended 90 or more days of school and scored in Level I or 
II on the 2000-01 pretest. 

Attrition reflects the number of students who participated in ALP but were removed from the 
analysis due to missing data on their posttest score. Attrition by grade is shown in Figure 21. 
Attrition was low (less than 5% in reading and less than 8% in math) across all grades. Students 
in 5‘*’ grade had the highest attrition rates. 



Students served by ALP were grouped into one of three categories of ALP timing: ALP service 
during the day only, outside the day only, or both during-and-outside the school day. Regression 
analysis was used to examine the impact of ALP -timing on posttest scores, controlling for 
pretest. Special Education, and FRL status. Separate regression analyses were run by grade and 
by subject (reading and math tests and ALP help). Directional findings are shown in Figure 21. 
More detailed results (e.g., beta weights) are available from E«&;R. 



Reading results indicated that the timing of reading help affected growth in 4*'’ grade, but not in 
grades 3 and 5. For fourth graders, students served in reading both during-and-outside the day 
scored lower than students served during the day only, controlling for pretest scores. Fourth 
graders served both during-and-outside the day scored 1.8 scale score points lower on the 
reading posttest. 




Negative effects were also found for Special Education and FRL students in all 3-5 grades. In 
other words, regardless of when ALP assistance was provided, in grades 3-5 Special Education 
and FRL students scored lower than non-Special Education or non-FRL students or students with 
relatively higher incomes, even when controlling for pretest scores. 



Results for math showed that the timing of math help did not affect growth in grades 3-5. In 
addition, FRL students scored lower on the posttest in 3rd and 4th grades. 

Overall, this analysis suggests that the timing of ALP service did not have a major impact on 
reading or math growth. However, when only the elementary schools with the highest and 
lowest growth were examined, timing did seem to make a difference, favoring schools that 
provided most ALP service outside of the school day (see next section). 
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Figure 21 

Posttest Regressed on Timing of ALP Help, 
Controlling for Pretest, Special Education, and FRL 



Explanatory 


READING 


MATH 


Variables 


Grade 3 


Grade 4 


Grade 5 


Grade 3 


Grade 4 


Grade 5 


Pretest 


+ 


+ 


+ 


+ 


+ 


+ 


Special Education 


- 


- 


- 


ns 


ns 




FRL 


- 


- 


- 


- 


- 


ns 


ALP Timing, 

During Only Vs: 

During and Outside 


ns 


_ 


ns 


ns 


ns 


ns 


Outside Only 


ns 


ns 


ns 


ns 


ns 


ns 


ALP Reading Participants: 


1,014 


638 


668 


469 


612 


298 


Attrition'^: 


16(1.6%) 


25 (3.9%) 


30 (4.5%) 


1 1 (2.3%) 


22 (3.6%) 


23 (7.7%) 


Final Regression Sample: 


988 


613 


638 


458 


590 


275 



Note 1: p<.05, ns=not significant. Significance p <.05 
+=positive significant impact 
- =negative significant impact 



Note 2: All students were eligible for and participated in ALP subject specified. Eligible students attended 90 or more days 
of school and scored in Level 1-2 on the 2001 Pretest. 

'^Attrition reflects the ALP eligible students who were removed from the regression analysis due to missing data 
posttest scores. 



Two-Year Followup 

All students who initially score below grade level do not make the kind of achievement 
improvements necessary to reach grade-level performance (Level III or IV) in one year. 
However, strong growth over more than one year can lead students to reach grade-level 
performance in two or more years. We therefore looked at the percentage of students reaching 
grade level over time for a cohort of students in grade 3 (in 1998-99) through 2000-02 (when 
they should have been in grade 5 if promoted both years). 

Progress was analyzed separately by subject (reading and math). Progress based on both level 
scores and scale scores was examined. We separated retainees from those not retained during 
this period. While we can report retention rates for both groups, sample sizes were too small to 
warrant further achievement analyses for retainees. 

Of the third graders who scored in Level I or II in the spring 1999, 13.6% of those who scored 
low in reading and 1 1 .9% of those who scored low in math were retained in grade 3 or 4. 
Patterns of performance for non-retainees are shown in the next figure. 
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• In reading, the percentage of students in ALP who reached grade level rose steadily by 
grade 4 (32%) and grade 5 (59%). (This increase was similar to middle school, which 
increased by 57%.) 

• In math, the percentage of students in ALP who reached grade level also increased, with 
the improvement shown by spring of fourth grade stronger than that shown in fifth grade. 
By 5**’ grade, nearly 69% of the students were able to reach grade-level performance 
(higher than in reading). (This increase was greater for elementary than middle school, 
which increased by 40%. ) 



Figure 22 

ALP Grade 3 Cohort: Percent Reaching Grade Level Over Time 




A second cohort study monitored the progress of students who scored in Level I or II in reading 
or math in the spring of 2000 at grades 3-8 (Baenen, 2003). Trends between spring 2001 and 
spring 2002 were examined by race. Reading results varied somewhat by race, with 37% of 
Black students and 48% of White students able to reach grade level scores by spring 2002. Math 
results did not vary as much by race, with 37% of Black and 42% of White students reaching 
grade level scores in spring 2002. 
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ALP 6-8 Programs 



PARTICIPATION 

Students scoring in Level I or II on the EOG in spring 2001, plus some Special Education 
students, were officially counted as eligible for the purpose of allocations. However, schools 
were free to invite low Level III students to participate as space allowed. The data collection 
method for middle schools did not change in 2001-02 (as the elementary level did). Scan forms 
listing all officially eligible students were sent to ALP coordinators, who were in charge of 
marking those served. Some involved classroom teachers; others did not. Over-reporting of 
students served was not as likely given this method. However, while the memo directions 
indicated that students served who were not on the list should be added, directions on the form 
itself did not specify this. Relatively few students were added; some under-reporting of Level III 
students served may have occurred. Data sheets returned for individual students indicated that 
56% of the 4,408 students eligible actually participated in the program. Most of those who did 
not participate received other WCPSS services. Participation by school varied from 27.6% to 
99.5% (see Attachment 2B). The following figure shows ALP participation in 2001-02. 
Additional detail by subgroup and overall is provided in Attachments 3A, 3B and 3C. 






Note: Eligibility for Grades 6-8 was determined by reading and math pretests on EOG in spring 

2001 . “Below Grade Level” for Grade 6 was determined by EOG reading and math pretests 
from spring 2001. Special education students without regular EOG scores were also 
considered to have needs for allocation purposes. These represent students counted in 
allocations. Any student who had a Level I or II score in either reading or math (or both) is 
included in “below grade level”. Students in “At or Above” had no Level I or II scores. 



Figure 23 

Student Participation in ALP 6-8 2001-02 



Grade 


Below Grade 
Level 


At or Above 
Grade Level 


Total Participants 


2001-02 


6 


791 


0 


791 


7 


935 


0 


935 


8 


733 


0 


733 


Total 


2,459 


0 


2,459 



Source: Data Capture Sheets May 2002; 
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